{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Caffe1 Models and Datasets\n",
    "\n",
    "This tutorial will help you acquire a variety of pre-trained models from the original Caffe repo, and translate these models to a format that Caffe2 expects. If you don't already have the Caffe repo, then clone it like so:\n",
    "\n",
    "```\n",
    "git clone https://github.com/BVLC/caffe.git\n",
    "```\n",
    "\n",
    "Start by importing the required modules."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "from __future__ import unicode_literals\n",
    "\n",
    "import os\n",
    "print(\"Required modules imported.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can setup your root folder for Caffe below if you put it somewhere else. You should only be changing the path that's being set for `CAFFE_ROOT`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# You should have checked out original Caffe\n",
    "# git clone https://github.com/BVLC/caffe.git\n",
    "# change the CAFFE_ROOT directory below accordingly\n",
    "CAFFE_ROOT = os.path.expanduser('~/caffe')\n",
    "\n",
    "# Make sure Caffe exists where you specified\n",
    "if not os.path.exists(CAFFE_ROOT):\n",
    "    print(\"Houston, you may have a problem.\") \n",
    "    print(\"Did you change CAFFE_ROOT to point to your local Caffe repo?\")\n",
    "    print(\"Try running: git clone https://github.com/BVLC/caffe.git\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's where you pick your model. There are several listed below such as AlexNet, GoogleNet, and Flickr Style. Uncomment the model you want to download."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pick a model, and if you don't have it, it will be downloaded\n",
    "# format below is the model's folder, model's dataset inside that folder\n",
    "\n",
    "#MODEL = 'bvlc_alexnet', 'bvlc_alexnet.caffemodel' \n",
    "#MODEL = 'bvlc_googlenet', 'bvlc_googlenet.caffemodel'\n",
    "#MODEL = 'finetune_flickr_style', 'finetune_flickr_style.caffemodel'\n",
    "#MODEL = 'bvlc_reference_caffenet', 'bvlc_reference_caffenet.caffemodel'\n",
    "MODEL = 'bvlc_reference_rcnn_ilsvrc13', 'bvlc_reference_rcnn_ilsvrc13.caffemodel'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a reminder, in Caffe, the deploy model is saved in two parts:\n",
    "\n",
    "    1) deploy.prototxt: contained the network architecture in human-readable protobuf format\n",
    "    2) .caffemodel file: contained the model weights and parameters for loading\n",
    "\n",
    "Therefore, to translate the model to Caffe2, we need both of these files. We already have the `deploy.prototxt` files for all of the models in `~/caffe/models`, so we need the learned weights.\n",
    "\n",
    "Below, we'll check to see if the `.caffemodel` file from the last model that we uncommented above already exists. If it does not already exist in the location that we specify, we will download it using the `download_model_binary.py` script in the Caffe repo. **Note that .caffemodel files are typically fairly large files, so downloading one will take a few moments.** We will be sure to print a message so we know when we can continue."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scripts to download the models reside here (~/caffe/models)\n",
    "# After downloading the data will exist with the script\n",
    "CAFFE_MODELS = os.path.join(CAFFE_ROOT, 'models')\n",
    "\n",
    "# this is like: ~/caffe/models/bvlc_alexnet/deploy.prototxt\n",
    "CAFFE_MODEL_FILE = os.path.join(CAFFE_MODELS, MODEL[0], 'deploy.prototxt')\n",
    "# this is like: ~/caffe/models/bvlc_alexnet/bvlc_alexnet.caffemodel\n",
    "CAFFE_PRETRAINED = os.path.join(CAFFE_MODELS, MODEL[0], MODEL[1])\n",
    "    \n",
    "# If the model folder doesn't have the goods, then download it\n",
    "# This is usually a pretty big file with the .caffemodel extension\n",
    "if not os.path.exists(CAFFE_PRETRAINED):\n",
    "    print(CAFFE_PRETRAINED + \" not found. Attempting download. Be patient...\\n\")\n",
    "    os.system(\n",
    "        os.path.join(CAFFE_ROOT, 'scripts/download_model_binary.py') +\n",
    "        ' ' +\n",
    "        os.path.join(CAFFE_ROOT, 'models', MODEL[0]))\n",
    "else:\n",
    "    print(\"You already have \" + CAFFE_PRETRAINED + \", skipping download...\\n\")\n",
    "\n",
    "# If the .prototxt file was missing then you're in trouble; cannot continue\n",
    "if not os.path.exists(CAFFE_MODEL_FILE):\n",
    "    print(\"Caffe model file, \" + CAFFE_MODEL_FILE + \" was not found!\")\n",
    "else:\n",
    "    print(\"Both the deploy.prototxt and .caffemodel files were found, ready to continue!\")\n",
    "    # Now we have init net and predict net .pb files to use"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have both the `deploy.prototxt` and `.caffemodel` files, we can translate the model to the Caffe2 saved model format, which consists of two serialized protobuf files:\n",
    "\n",
    "    1) init_net.pb\n",
    "    2) predict_net.pb\n",
    "    \n",
    "To do this, we will use Caffe2's translator script at `~/caffe2/caffe2/python/caffe_translator.py`.\n",
    "\n",
    "**Again, depending on the size of the model, this may take a minute or two**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the CAFFE2_ROOT\n",
    "CAFFE2_ROOT = os.path.expanduser('~/caffe2')\n",
    "init_net_out = os.path.join(CAFFE_MODELS, MODEL[0], 'init_net.pb')\n",
    "predict_net_out = os.path.join(CAFFE_MODELS, MODEL[0], 'predict_net.pb')\n",
    "\n",
    "# Run the caffe_translator.py script to translate to Caffe2 if files do not already exist\n",
    "if (not os.path.exists(init_net_out)) or (not os.path.exists(predict_net_out)):\n",
    "    print(\"Protobuf files not found. Running translation. Be patient...\\n\")\n",
    "    os.system(\n",
    "        'python' + ' ' + os.path.join(CAFFE2_ROOT, 'caffe2/python/caffe_translator.py') +\n",
    "        ' ' + CAFFE_MODEL_FILE + ' ' + CAFFE_PRETRAINED + ' ' + \n",
    "        '--init_net' + ' ' + init_net_out + ' ' +\n",
    "        '--predict_net' + ' ' + predict_net_out\n",
    "    )\n",
    "else:\n",
    "    print(\"You already have both .pb files, skipping translation...\\n\")    \n",
    "\n",
    "# Print if files are where they are expected to be\n",
    "if (not os.path.exists(init_net_out)) or (not os.path.exists(predict_net_out)):\n",
    "    print(init_net_out + \" and/or \" + predict_net_out + \" was NOT FOUND!\")\n",
    "else:\n",
    "    print(\"Protobuf files can be found at: \\n\", \n",
    "              os.path.join(CAFFE_MODELS, MODEL[0])), \"!\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, we have translated the model from Caffe to a format that Caffe2 can use. Have a look at our other tutorials, such as *Loading Pretrained Models* to see an example of how to use these .pb files for inference."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
